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Abstract — We study LDPC codes for the channel with 2™- 
ary input x e GF(2)"' and output y = x + ze GF(2)'". The 
receiver knows a subspace V C GF(2)"' from which z = y — x 
is uniformly chosen. Or equivalently, the receiver receives an 
affine subspace y ~V where x lies. We consider a joint iterative 
decoder involving the channel detector and the LDPC decoder. 
The decoding system considered in this paper can be viewed 
as a simplified model of the joint iterative decoder over non- 
binary modulated signal inputs e.g., 2'" -QAM. We evaluate the 
performance of binary spatially-coupled MacKay-Neal code by 
density evolution. EXIT-like function curve calculations reveal 
that iterative decoding threshold values are very close to the 
Shannon Umit. 



I. Introduction 

Spatially-coupled (SC) low-density parity-check (LDPC) 
codes attract much attention due to their capacity-achieving 
performance and a memory-efficient sliding-window decod- 
ing algorithm. The studies on SC-LDPC codes date back 
to the invention of convolutional LDPC codes by Felstrom 
and Zigangirov [1]. They introduced a construction method 
of (c?i, (ii)-regular convolutional LDPC codes from {di,di-)- 
regular block LDPC codes [2]. The convolutional LDPC codes 
exhibited better decoding performance than the underlying 
block LDPC codes under a fair comparison with respect 
to the code length. Lentmaier et al. observed that (4,8)- 
regular convolutional LDPC codes exhibited the decoding 
performance surpassing the BP threshold of (4,8)-regular block 
LDPC codes [3]. Further, the BP threshold coincides with the 
MAP threshold of the underlying block LDPC codes with a 
lot of accuracy. Constructing convolutional LDPC codes from 
a block LDPC code improves the BP threshold up to the MAP 
threshold of the underlying codes. 

Kudekar et al. named this phenomenon "threshold sat- 
uration" and proved rigorously for the binary-input erasure 
channel (BEC) [4] and the binary-input memoryless output- 
symmetric (BMS) channels. [5]. In the limit of large d\,dr,L 
and w, the SC-LDPC code ensemble {d\,di-,L,w) [4] was 
shown to universally achieve the Shannon limit of BMS 
channels under BP decoding. This means the transmitter does 
not need detail statistics of the channel but needs to know only 
the channel capacity. Such universality is not supported by 
other efficiently-decodable capacity-achieving codes. In fact. 



polar codes [6] and irregular LDPC codes [7]. According to the 
channel, polar codes need frozen bit selection [6] and irregular 
LDPC codes need optimization of degree distributions. We not 
that recently Aref and Urbanke proposed SC rateless codes 
[8] which are conjectured to universally achieve the capacity 
of BMS channels without knowing even the capacity of the 
channel at the transmitter 

MacKay-Neal (MN) codes [9] are non-systematic two-edge 
type LDPC codes [10], [2]. The MacKay-Neal codes are 
conjectured to achieve the capacity of BMS channels under 
ML decoding. Murayama et al. [11] and Tanaka et al. [12] 
reported the empirical evidence of the conjecture for BSC 
and AWGN channels, respectively by a non-rigorous statistical 
mechanics approach known as replica method. In [13], Kasai 
and Sakaniwa presented a spatial coupling method of SC- 
MN codes. Empirical results showed that SC-MN codes with 
bounded density achieve the capacity of the BEC. It was 
observed that the SC-MN codes have the BP threshold close 
to the Shannon limit. 

In this paper, we study coding over the channel with 2™- 
ary input x € GF(2)'" and output y G GF(2)'". The receiver 
knows a subspace V C GF(2)™ from which z = y — x_\s 
uniformly chosen. Or equivalently, the receiver receives an 
affine subspace y — V := {y — z\ z^V} \n which the input 
X is compatible. This channel model is used in the decoding 
process for network coding [14] after estimating noise packet 
spaces. In [14], non-binary LDPC codes are used, which 
results in high-decoding complexity 0{m?) at each channel 
factor node and parity-check nodes. We do not use non- 
binary codes but binary codes for coding the non-binary input 
channel. Furthermore, we consider the joint iterative decoding 
between channel the channel detector and the code decoder 
The channel detector calculates LLR of the transmitted bits 
from a channel output and messages from the BP decoder 
Such a decoding system can be viewed as the simplest model 
of joint iterative decoders which involve the channel detector 
of non-binary modulation (e.g., 2™-QAM) and the BP decoder 
for LDPC codes. 

The aim of this paper is to evaluate the joint iterative decod- 
ing performance of binary SC-MN codes over the non-binary 
input channels. To this end, we use density evolution (DE) 
analysis for joint iterative decoding of binary SC-MN codes. 
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Fig. 1. Factor graph representation of SC-MN codes with di = 4, dr = 2, dg = 2 over CD(m = 2, e) or BD(m = 2, e). 



EXIT-hke function curve calculations reveal that threshold 
values are very close to the Shannon limit. 

II. Channel Model 

In this paper, we consider channels with input x_ G GF(2)™ 
and output y = x + z e GF(2)™, where z € GF(2)™ is 
uniformly distributed in a linear subspace V C GF(2)'". We 
refer to this subspace as noise subspace. The dimension of V is 
distributed as Pr(dim(V^) = d) =: pd- Given the dimension d, 
V is also uniformly distributed, i.e., Vt:{V — v\ dim(T^) ~ 



d) = where 



";j = nf=o is a 2-Gaussian 

binomial. The normalized capacity per input bit is given by 
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The latter part equals to H{Z_) which is independent of p(X) 
and former part is maximized when 2L is uniformly distributed. 
Hence, it follows that the normalized channel capacity is given 

We consider two types of smooth channels defined by pd- 
The simplest dimension distribution is deterministic one. The 
channel W(m, w) for G {0, 1, . . . , to} is defined by 



Pd 



{d = w) 
(d^w) 



W(to, w) for large m was used in a decoding process of the 
network coding scenario [14]. In [14], the data part of each 
packet is represented as x G GF(2)'". Packets are coded by 
non-binary LDPC codes whose parity-check coefficients are in 
the general linear group GL(to, GF(2)). The noise subspace 
V is estimated by padding zero packets and using Gaussian 
elimination. Note that, in this paper, unlike [14] we use binary 
LDPC codes for coding the 2™-ary input channel. Note also 
that, our DE calculation is currently limited for small to. 

In order to evaluate the performance of a give coding and 
decoding system, we prefer smoothly parametrized channels. 
Instead of W(m, w), let us consider CD(m, e) as a smoothly 
parametrized of W(e), where CD stands for concentrated 
dimension. 

em — [emj {d = [emj), 
Pd '■— ^ 1 ^ + [£ni\ {d = [emj - 
(otherwise). 
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For TOe = w, W(to, w) is identical to CD(m, e). The normal- 
ized capacity of CD(to, e) is given by 1 — e. 

Another interesting non-binary input channel is BD(m, e) 
whose dimension distribution is given by the binomial distri- 
bution Pd := (™)e'^(l — e)™^'^, where BD stands for binomial 
distribution. This channel is realized by multiplying random 
TO X m binary non-singular matrix A by the input bits x 
before transmitting through the binary erasure channel with 
erasure probability e [15]. The affine subspace is given by 
{x : AeS^e = j4£;cx|Jc}, where E C {1, . . . , to} is the set of 
erased output indices, Ae denotes the submatrix of A indexed 
by the elements of E, and x^ is the corresponding subvector 
The normalized capacity of BD(to, e) is given also as 1 — e. 

III. Spatially-Coupled MacKay-Neal Codes 

Let N denote the number of channel uses for one codeword, 
in other words, at each channel use, a TO-bit symbol is trans- 
mitted through the channel. The total number of transmitted 
bits are n := toA^. We use SC-MN codes of length n. 

In this section, we define binary SC-MN codes. We define 
a (di, dr, dg, L, w) SC-MN code ensemble as follows. The 
Tanner graph of a code in the {d\,d,-,dg,L,w) SC-MN code 
ensemble is constructed as follows. At each section i G 
Z := {. . . , -2, -1, 0, +1, +2, . . . }, consider bit nodes 
of degree d\, AI bit nodes of degree dg. We refer to those 
two types of bit nodes as type 1 and type 2, respectively. 
Additionally, at each section i, consider M check nodes which 
adjacent to d^ bit nodes of type 1 and dg bit nodes of type 2. 
Connect randomly these nodes in such a way that for i G Z 
and j = 0, . . . , w — 1, bit nodes of type 1 at section i and 
check nodes at section i + j are connected with edges 
and bit nodes of type 2 at section i and check nodes at 
section i + 7 are connected with ^^^^ edges. Shorten the bit 
nodes of type 1 and 2 at section |i| > L, in other words, 
set the bits to zero and do not transmit them. Puncture the 
bit nodes of type 1 at section \i\ < L, in other words, the 
bits are not transmitted. Hereafter, we call bit nodes of type 
1 and 2, punctured and transmitted bits, respectively. Note 
that this code ensemble is nicely represented as LDPC codes 
defined by a joint degree distribution [16]. The definition of 
the (di, dr, dg, L, w) SC-MN code ensemble is based on that of 
(di, di , L, w) randomized SC-LDPC code ensemble. For more 
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details on {d\, d^, L, w) randomized SC-LDPC code ensemble, 
we refer the readers to [4, Section II. B]. 

Denote the number of transmitted bit nodes, punctured 
nodes by Vt, Vp, respectively, n = Vt ^ LM, Vp = 
^LM, where L := 2L + 1. The number of check nodes 
of degree at least 1, denoted by Vc, can be counted by 
the same way as in [4, Lemma 3] as follows. Vc ~ 
M[2L -w + 2E^^o(l - (i)'^^(i The design cod- 
ing rate R^^ {di,d,, L,w) is given by i?"'^^, d, , L, u;) := 

Vt+Vp-V. _ I l+»;~2Er^o(l-(t)'''(i)'''') _ 

oo). 



IV. Decoding Algorithm 



In this section, we define a factor graph [17] for decoding 
the SC-MN code over the channel. Denote a {di, d^, dg, L, w) 
SC-MN code by C. Denote the transmitted code sym- 
bols, punctured bits and the received symbols by a; = 
{xi,...,XmN) =■■ (xi, • • • jXat). 2 = (zi, . . . ,zvp) and y = 
(li,---,YAr), respectively. 

We adopt the sum-product algorithm [17] over a factor 
graph defined by the following factorization of P{x,z\y). 

P{x,z\y) = P{y\x,z)P{x,z)/P{y), 

N 



P{y\x,z)=l[P{y^\x,), 



4=1 



P{y,k^ 







(otherwise) , 



(1) 



(2) 



pix, z) = i[i[j2 + E = 

fe=l j£dtk j&9pk 

where dtk (resp. dpk) is the set of indices for transmitted (resp. 
punctured) bit nodes adjacent to the fc-th check node of C. 
Figure [T] shows an example of a factor graph representation of 
SC-MN codes with di = 4, d, = 2, dg = 2 over CD(m = 2, e) 
or BD(m = 2, e). 

The calculation for message ^i^im+j{xim+j) from the 
factor node i of ([T]! to the variable node Xm+i is given by 



E 



.eGF(2) 



I n 

J=2 



where /i 



{xim+j ) is the sum-product message from the 



A. Factor Node of Channel 

The messages stay in {0, 1, ?} during the entire decoding 
process. DE update equation for the check node is obvious. 
First, we will focus on deriving the DE for the factor node 1 
of O 

Rathi and Urbanke [18] developed the DE of non-binary 
LDPC codes for the EEC. The DE allows us to track proba- 
bility mass functions of the dimension of the linear subspaces. 
For ^ > 0, the DE tracks the probability vectors P'^' — 
(Pq^\ . . . , Pm) £ [0, 1]™+! which are referred to as densities. 

Without loss of generality, we can assume all-zero code- 
words were sent. We fix the first one of m transmitted bit 
nodes 1, . . . , m adjacent to the factor node 1 and derive the 
probability that the outgoing message n along the edge is 
erasure. There are m — 1 incoming messages to 1. Denote 
the probability that a randomly picked message from the 
transmitted bit nodes are erased by z. Denote the indices of 
erased messages by /C. Note that 1 ^ /C. Define a subspace 
Vi ^ {x = {xi,...,Xm) e GF(2)™ \ x^ = i K.} and 
denote Di := dim(Vi). We have /C = m — Di Then, it follows 
that 



P\^l{^ + l) 







)2*(1 - z)'"-^-i 0<i<m-l 
i = -1. 



Let Vq C GF(2)™ be the noise subspace which is known 
at 1. The outgoing message is not erased if and only if 
all vectors in V2 Vi D Vq have at the first index, 
in precise, "^{xi, . . . , Xm) & V2,xi = 0. From [18], the 
probability PD;\Do.Di{k\j,i) that V2 has dimension k given 
Vi has dimension i and Vq has dimension j is given as follows. 



I32|Ai,-D 



Since bits whose indices in K, are known, i.e., 0, D2 is 
distributed over index support {!,..., m} \ K.. The outgoing 
message n is known if the support of V2 does not include 
1. By counting the number for subspace of dimension i and 
i — 1 over the support of size k, we obtain the probability 
Pfj,\D2,Di{T\k,'i) that outgoing message is ? given that V2 has 
dimension k and Vi has dimension i is given as follows. 

■1 -1 



P 



,{?\k,z) = l- 



i - 1 
k 



Therefore, we have the probability fc.m{z) '■— P{pi. — ?) that 
outgoing the message is ? as follows. 



variable node of Xim+i to i. At the beginning of the algorithm, f^^^n^z) = ^ ^ Pp|£)2^i3j (? |fc, i)PD2,Di{k, i) 



(M™+,^i(0),M™+,^j(l)) is set to (1/2,1/2). From [2, 
Chap. 2], we know that if (m,,„+j^5(0), is either 

:= (1,0), 1 := (0,1) or ? (1/2,1/2), (x„„+, ) 

is also either 0,1, or ?. This message calculation at i is 
efficiently accomplished by Gaussian elimination with 0{riT') 
calculations. Since the factor node for each factor in (|2]i is a 
check node. Hence, the messages stay in {0, 1, ?}. 

V. Density Evolution Analysis 

In this section, we derive DE equations for the sum-product 
algorithm over the factor graph defined in the previous section. 



fc=0 i=k 

PD2,DAk,l)^ E PD2\Do,DAk\j,^)PDo{j)Pl^'!il) 
J=k 

B. Density Evolution for Spatially-Coupled MacKay-Neal 
Codes 

Let PI ' and q- be the erasure probability of messages 
emitting from punctured bit and transmitted bit nodes to check 
nodes, respectively, at section i at the i-ih round of BP 
decoding in the limit of large M. Similarly, let be the 
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TABLE I 

Threshold VALUES dr, L, w) for (di 



: 4,(ir = 2,dg = 2,L,-u;) 



SC-MN CODES FOR CD(m, e) AND BD(m, e). 





CD(m, e) 


BD(m, e) 


m 


L = lO.u) = 2 


L = 20, «; = 3 


L = 10,«i = 2 


L = 20, m; = 3 


1 


0.49998527 


0.49999998 


0.49998527 


0.49999998 


2 


0.49950900 


0.49999936 


0.49987196 


0.49999983 


3 


0.49913150 


0.49999850 


0.49954538 


0.49999942 


4 


0.49714179 


0.49998625 


0.49885380 


0.49999741 


5 


0.49566948 


0.49997243 


0.49768392 


0.49999097 


6 


0.49166023 


0.49989196 


0.49596851 


0.49997517 



erasure probability of messages emitting from channel factor 
nodes to punctured bit nodes at section i at the ^-th round of 
BP decoding in the limit of large M. DE update equations 
of the randomized (rfi, dr, dg, L, w) SC-MN code are given as 

0. For \i\ < L, 



follows. For \i\ > L and £ > 0, 'pf' = 

pf^ = = 1- For |i| < L, 

{e+i) 
Pi 

w-l w-1 U) 



i=0 



^ w—l 

w 



w-l (t) 

^ ^-^ w 

w-l U) w-l U) 

^ ^ W ^ W ' ^ 

k=0 k=Q 



-1 

1 



C. Threshold Values 
We define the threshold value as 



e* sup {e £ [0, 1] 

se[o,i] 



lim pI^' = 0, i = 



-L, 



for the channel CD(m, e) or BD(to, e). In words, the SC-MN 
codes enable reliable transmissions over the channel CD(m, e) 
or BD(m, e) if e < e*. 

Threshold values e*{d\,dr,L,w) for {d\ = 4, dr = 2,(ig = 
2, L, w) SC-MN codes are numerically calculated and listed 
in Tab. U As was also observed in the case of BEC [13], [4], 
we also observed the the threshold values remain almost the 
same for large L e.g. 200 for fixed d\, d^, dg, w. Since the rate 
of {d\ = 4, dr = 2,(ig = 2,L,w) SC-MN codes converges to 
di-/d\ = 1/2 in the limit of large L, we ignore the rate-loss. As 
can be seen, the threshold values are very close to the Shannon 
limit e = 1/2 of {d\ = 4,(ii. = 2,dg = 2,L = oo,w < oo) 
SC-MN codes. Interestingly, threshold values are degraded as 
TO increases. This degradation is cased by wiggles as seen also 
in the case of BEC [4]. The gaps to the Shannon limit seems 
to grow exponentially with to. 




0.4990 



0.5005 



0.5010 



Fig. 2. EBP EXIT-like curve of (dj = 4,dr = 2, dg = 2, L,to = 2) 
SC-MN codes with L = 10 and L = 20. The channel is CD(m = 2, e). 
The threshold for each code is given by erasure probabiHty e (X-axis) of the 
leftmost point on the curve. The threshold values are almost the same around 
e = 0.499509 for L = 10 and L = 20. 



m , (i+i) (i) 
Pl and Qi = ql = 



qi=0 for \i\ > L, qi = pi 
for \i\ < L, For any e G [0, 1], a fixed point {e,p ^ 0,q = 0) 
is called trivial. Trivial fixed points correspond to successful 
decoding. We define EBP EXIT-like curve as the projected 
plots (e, /i^^^(e)) of fixed points {e,p,q), other than trivial 
ones, onto the following EXIT-like function. 



;EBP 



In Fig. 121 we plot the EBP EXIT-like curve of {d\ = 
4,4 = 2,dg = 2,L,w ^ 2) SC-MN codes for CD(to, e). 
As can be seen, the threshold values are almost the same 
around e = 0.499509 for L = 10 and L = 20. In Fig. [3] 
and Fig. g] we plot the EBP EXIT-like curve of {d\ = 
4, dr = 2, dg = 2,L,w = 2) SC-MN codes for CD(m, e) 
and BD(7Ti, e), respectively. From those curves, it can be seen 
that the threshold degradation is caused by large wiggles. The 
wiggle size seems to grow exponentially with m. This problem 
would be serious if to got large. In fact, large m is assumed 
in the network coding scenario [14]. However, wiggles are 
significantly mitigated by increasing randomized window size 
w as in the right column of Fig. [3] and Fig. |4] 



VII. Conclusion 



VI. EXIT-LIKE Function 

In this section we investigate the reason of threshold degra- 
dation by an EXIT-like function. By "EXIT-like", we mean 
that the area theorem [2] does not necessarily hold. However, 
EXIT-like function allows us to understand how the decoding 
performance is affected by the fixed points of DE. 

Consider fixed points of the DE system described in Section 
IMl i.e., (p := {Pi)r=-oo.(l - fe).=-oo:e) such that p, = 



In this paper, we evaluated the performance of binary SC- 
MN codes for the channels with affine space outputs. We de- 
rived DE equation and observed the threshold values are very 
close to the Shannon limit. We conclude that binary spatially- 
coupled codes are not only universal for binary input channels 
but likely universal also for non-binary input channels. In the 
future work, we will extend this work for channels with non- 
binary modulated signal inputs with AWGN. 




Fig. 3. Left: EBP EXIT-like curve of SC-MN codes with L = 10,w = 2, riglit: EBP EXIT-like curve of SC-MN codes with L = 20,w = 3, The channels 
are CD(m, e) for m = 1, ... ,6. The tlireshold for each code is given by the most left erasure probability e on the EBP EXIT-hke curve. Unfortunately 
and interestingly, wiggles are much amplified by increasing channel input size 2'". However, wiggles are significantly mitigated by increasing randomized 
window size w. Note that scale is magnified 100 times from left to right. 
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Fig. 4. Left: EBP EXIT-like curve of {di = 4, d,. = 2, dg = 2, L, to) SC-MN codes with L = 10, to = 2, right: EBP EXIT-like curve of SC-MN codes 
with L = 20, w = 3, The channels are BD(m, t) for m = 1, . . . , 6. Wiggles are smaller than those of CD(m, e) for the same di, dr, dg, L and w. 
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